Papers by Matan Ben Noach
Transfer Learning Between Related Tasks Using Expected Label Proportions (D19-1)
Copied to clipboard
| Challenge: | Existing methods of data supervision are limited by labeled training data. |
| Approach: | They propose a method where models are trained based on expected label proportions. |
| Outcome: | The proposed method improves on a sentence-level sentiment predictor and is cumulative with LM-based pretraining. |
Compressing Pre-trained Language Models by Matrix Decomposition (2020.aacl-main)
Copied to clipboard
| Challenge: | Large pre-trained language models reach state-of-the-art results when fine-tuned individually; They also come with a significant memory and computational requirements, calling for methods to reduce model sizes (green AI). |
| Approach: | They propose a two-stage model-compression method to reduce a model’s inference time cost by decompressing the model into smaller matrices and performing feature distillation on the internal representation. |
| Outcome: | The proposed method reduces the number of parameters by 0.4x and increases inference speed by 1.45x while preserving the information contained within the model. |